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#deeptiktok ... peppaaaa #deeptiktok ... PLEASE #fyp #deeptiktok... 


Deeptiktok 


DeepTok, also known as Deep TikTok, is a subgenre of Alt/Elite TikTok, 
Definition: Alt TikTok, also known as Elite TikTok is a TikTok subculture featuring 
eccentric, alternative content. Alt TikTok, also known as Elite TikTok is a TikTok 
subculture featuring eccentric, alternative content. 5 Aug 2020 
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1. FOCUS ON ‘ALTERNATIVE’ TIKTOK 

2. DISCUSS THREE METHODS FOR 
WORKING WITH COLLECTIONS OF 
SOCIAL VIDEO CONTENT 

3. COMBINE TIKTOK METADATA WITH 
Al-DRIVEN VIDEO & AUDIO ANALYSIS 


UNDERSTANDING TIKTOK VIDEOS 


1. TikTok videos are short. 2. TikTok is all about vertical video. 3. Many videos 


tend to be music focused and use original sounds. 4. TikTok has an extensive 


library of sound templates. 5. Many videos also apply searchable sound and 


video effects in combination with personalized text/emoji stickers. 


-> TikToks are variously networked/searchable/templatable 


Engaging logic of TikTok video memes 


TikToks are variously networked/searchable/templatable. The main networker is often the sound, not the hashtag. 


Platform Metadata 


interactive features 
(likes, views, shares, comments, 
challenges, duets etc.) 
+ 
timestamps, accounts, hashtags, sounds, 
captions, stickers, effects, etc. 
+ 


moving images/audio content 


Tiktok video length can range from 3 seconds to 3 minutes. 


60fps 


One second 


30fps 


The usual frame rate is 30 frames per second -> 1 second of video generates 30 images. 


TIKTOK: THE SOCIAL MOVING IMAGE 


1.Finding the right sampling technique for frame 


extraction and detection of visual elements 


On sampling and visualization techniques see e.g., Manovich 2013, on video analysis with Al see e.g., Arnold et al. 2019) 


&t Video Intelligence API Visualiser https://zackakil.github.io/video-intelligence-api-visualiser/ 


Try with your data 


You can analyse a video by following 
the instructions here. Also see the script 
here if you want to run all features at 
once. 


This app doesn't send or store any of 
your video or annotation data. It just 
visualises local data files. 


Your video 


Choose file | No file chosen 


Your json 


Choose file | No file chosen 


Label Detection @ Object Tracking @ Person Detection @ Face Detection @ Logo Recognition @ Speech Transcription @ Text Detection @ | Explicit Content Dete 


| 0.00s — (6.80s) —> 6.80s | 6.84s — (3.17s) > 10.01s | 10.05s — (4.38s) —> 14.43s 14.478 — (1.678) > 16.14s 16.18s — (3.13s) — 19.315 [ 19.35s — (1.718) —> 21.06s } 21.10s — (1.718) —> 22.815 22.86s — (1.548) —> 24.40s 
24.448 — (2.17s) —> 26.61s 26.65s — (3.25s) —> 29.90s 29.95s — (1.92s) —> 31.87s | 31.91s — (3.55s) > 35.45s 35.49s — (3.42s) —> 38.91s 38.96s — (2.92s) > 41.88s 41.92s — (3.09s) —> 45.00s 45.05s — (4.30s) —> 49.34s 


[ 49.385 — (11.09s) —> 60.48s | | 60.52s — (10.34s) —> 70.86s | 70.90s — (1.96s) —> 72.86s | | 72.918 — (1.298) —> 74.20s | | 74.24s — (3.09s) —> 77.33s | 77.378 — (2.928) — 80.29s } | 80.33s — (7.47s) — 87.80s | | 87.84s — (8.93s) —> 96.76s | 


| 96.81s — (6.17s) > 102.98s | 103.02s — (9.55s) > 112.57s | 112.61s — (4.92s) — 117.53s 117.58s — (3.04s) — 120.62s | | 120.66s — (2.54s) —> 123.21s | | 123.25s — (2.59s) —> 125.83s | | 125.888 — (4.21s) —> 130.09s | | 130.13s — (2.42s) —> 132.55s } 


| 132.59s — (2.46s) —> 135.05s | 135.09s — (1.178) —> 136.26s | [ 136.30s — (0.71s) — 137.01s | 137.05s — (1.58s) —> 138.64s | [ 138.68s — (2.13s) > 140.81s | | 140.85s — (1.798) —> 142.64s } 142.688 — (2.46s) —> 145.15s | [ 145.19s — (4.50s) —> 149.69s 


| 149.73s — (1.04s) > 150.78s 150.825 — (2.00s) > 152.82s | [ 152.868 — (1.33s) > 154.20s | 154.245 — (2.218) > 156.45s | [ 156.495 — (1.96s) > 158.45s | | 158.49s — (0.79s) —> 159.28s | 159.33s — (0.96s) > 160.295 | [ 160.338 — (2.67s) > 163.00s 


Google Video Intelligence API, for example, extracts and annotates the middle frame of each shot based on its visual content. The 
method is very powerful when it comes to complex story arcs and multiple shot transitions. Limitation: Web detection is not included. 


Shot detection Label detection Google Video Al - Video Content Analysis 


0.00s-15.015s hair; long hair; dreadlocks; black hair 


Many TikToks are filmed in one shot. 


15 seconds of this video generate 450 frames. Frame rate = 30 fps. 
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Google Video Al label detection provides the labels of the individual objects present in a 
given video. However it does not contain additional web information, which is crucial for 
understanding the social aspects of video distribution and TikTok virality. 


(...) Web detection detects web references to an image based on content from pages with similar images.” (Google Cloud 


Vision API + Memespector GUI (Chao 2021) + see e.g., Hagen 2017; Omena et al. 2021; Pilipets 2021; Rogers et al. 
2021; Pearce et al. 2022; Burkhardt & Rogers 2022; Tucci 2022: on GV labels see e.g., Geboers & Van de Wiele 2020). 


=~ ‘C)) “Google Vision API labels detect and extract information about entities in an image, across a broad group of categories. 


15 sec/15 frames 


Best Guess Labels dreadlocks; tiktok bored in the house; bored in the house curtis roach; rap challenge 


Web entities Labels 

Bored In The House; Curtis Roach; Tyga & Curtis Roach; Detroit; Hand; Hairstyle; Arm; Beard; Flash photography; Gesture; Wood; 
Dreadlocks; Blond; Hair; Brown hair; Bangs; Long hair; TikTok; Song; Eyewear; Fashion design; Facial hair; Fun; Event; Flooring; Eyebrow; 
SoundCloud; Meme; YouTube; Spotify; Musician;Hip hop music; Rapping Facial expression; Table; Finger; Dreadlocks; Long hair 


Pages with full matching images 


clickondetroit.com; mashable.com; youtube.com; ispot.tv; instagram.com; revolt.tv; inkvilla.com; energetikas.It; materielesthetique.fr; spotify.com; 
capitalfm.com; facebook.com; buzzfeed.com; eska.pl; zip.news; dexerto.com; nytimes.com; forkast.news; theshelf.com; twitter.com; cbc.ca 


Settings 


Google Vision Features Plecte Credential file (Documents/Memespector-GUl-macOS/google-cloud-token.|son Browse... 


© Microsoft Azure Features | 7 Endpoint | >gnitiveservices.azure.com/ Subscription key abcdef0123456789 


| Clarifai Model 
©) Open Source Model 


Image Sources 


Add image files on this computer 


Output 
JSON file (full results) 


CSV file (simplified results) 


API key abcodel0123456789 


ImageNet) Endpoint https://europe-westi-d : ods-resources.cloucfunctions.net/classify imag 


a foider containing images on this computer a text file containing image locations images on the web 
Pp 9 9 9 


See https://github.com/jason-chao/memespector- 


/Users/Use ocuments/Memespector-GUl-macOS/cv-apis-date_time.json Browse... 


/Users/User/Documents/Memespector-GUI-macOS/cv-apis-date_time.json Browse... 


How can we analyze a collection of videos? 


FLAC FLAC 001_mp4_audio.flac 
——— Jd ae 
001_mp4_frame0001.jpg 
001_mp4_frame0002.jpg 
001_mp4_frame0003.jpg 
002_mp4_frame0001.jpg 


002_mp4_frame0002.jpg 
002_mp4_frame0003.jpg 


Extracted video frames and audio tracks can be analyzed with different Al features. 


& Video Frame and Audio Extractor.ipynb 


File Edit View Insert Runtime Tools Help Changes will not be saved 


+ Code + Text # Copy to Drive 


~ Video Frame and Audio Extractor 
This tool extracts the frames and audio tracks from video files. It is designed to work well with video files stored in Google Drive. 


Input 


* Video fil 
Video files Video Extractor (Chao 2023) offers different sampling 


Output techniques for extracting frames (stills) from a folder of 
input video files. It saves extracted images in a new 
* Frames (stills) of the input video files folder, which then can be analyzed with different 
* Audio tracks (in flac format) of the input video file: computer vision techniques. It also extracts audio tracks 


for speech-to-text recognition. 


& Speech-to-Text Converter.ipynb 


File Edit View Insert Runtime Tools Help Changes will not be saved 


+ Code + Text S Copy to Drive 


~ Speech-to-Text 


This tool converts speech in audio files to text using the Google Cloud Speech-to-Text API. It is designed to work well with audio files stored in 
Google Drive. 


Input 
Speech-to-text converter (Chao 2023) transcribes audio 
content as captions using Google Cloud Al. It saves 
Output recognized text in a spreadsheet file, which then can be 
merged with TikTok metadata. The main affordance of this 
+ The speech in text tool for TikTok research is that it (at least partially) helps to 
work around the problem of the “original sound”. 


* Audio files (in flac format) 


G H 


data/stats/c data/stats/p data/effectS data/music/authorh data/musicititle 


Audio content: TikTok creators 
often upload their videos with 
“original sound” instead of using 
TikTok’s built-in music library. If 
the video is linked with a song 
taken from the TikTok library, 
the song and author name is 
listed. However, as soon as 
content involves audio editing, 
the sound will be listed as 
“original sound” and provided 
with a unique numeric ID. —> 
The challenge of sound 
recognition/sense making. 


1380 488100 


1666 175800 


Spreadsheet output with TikTok 
metadata scraped using 
Zeeschuimer (Peeters 2021) 
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For more on TikTok sounds see e.g., Abidin & Kaye 2021; 
Bainotti et al 2022, Geboers et al. 2022, 


Networked soundscapes 
Resonant sounds and co-hashtags 


Networked soundscapes 


original sounds 
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= #deeptiktok 


co-hashtags 


960400 #deeptiktok #viral #fyp #darkside #alttiktok #alt #deep #foryoupage 
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NAVIGATING THE VERNACULAR STYLE SPACE OF 
#DEEPTIKTOK BY CHOOSING DIFFERENT PATHS: 


SOUNDS, EFFECTS, EMOJIS, HASHTAGS... 
af 
AUDIOVISUAL CONTENT/TEMPLATES 


digital (moving) images as data multiplicities 


Platform Metadata Computer Vision Data 
engagement metrics e.g., Google Vision API Labels 
(likes, reblogs, comments, etc.) (objects, colors, bodies, actions, etc. ) 
+ + 
hashtags, timestamps, account names, detection of web entities and pages 


. ’ jpg .png .gif, .mp4, etc. 
captions, sounds, stickers, effects, etc. with matching images; speech detection 


How to contextualize based on research questions and using different 
visualization layouts (plot, stack, grid, network, montage, matrix, etc)? 


#deeptiktok | 951 videos | 18199 frames 


Deep TikTok is a branch of Alternative or Elite TikTok. While Alt TikTok highlights the independent nature of diverse categories, 
DeepTok sticks with a handful of ultra-original, niche topics, and is known for its deepfried aesthetic. 
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Effect templates 


video content and co-hashtags 
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According to i-D, dreamcore is defined as an aesthetic that 
revolves around weird imagery “and objects that often 
tap into nostalgia or things you may have been in 
childhood.” But instead of being “cute or comforting,” it 
creates a sense of surrealism and uneasiness, and most of 
the videos feature slowed down music ... 18 Jun 2021 


https:/Awww.ypulse.com > Articles 
Dreamcore is the Latest Aesthetic Taking Over TikTok on The ... 


@ About featured snippets + MB Feedback 
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| Russian tre’ cult #Deeptiktok: Cross-platform riffing and 
o\* vernacular engagement 
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weirdcore and creepypasta 
#fyp and #cursedimages 


#areyoucomingtothetres: 


body-image dominated 
#cometothetree vernacular 


© not too many co-hashtags are shared 
between platforms: #tree and #alttiktok 
being notable exceptions 


© riffing off and repurposing takes place via 
color customiser effects, stickers, 
e and distorted body images 


© YT shorts and TikToks share are located 
within the same semantic neighbourhood 
sharing web references to weirdcore and 
creepypasta genres 


close-up videos 
with stickers and color filters 


domains matching content 


from several Instagram videos #Deeptiktok: Multi-situatedness and 
web distribution 


domains with images matching 
content from all five platforms 


© sound ‘covers’ of weirdcore tunes on 
soundcloud match cursed images 
a / also used in TT and YT videos 


© static cursed and creepypasta images 
er stiched together in YT and TT videos also 
appear on Tumblr and reddit (where they 
| : were established as a vernacular genre 


© Instagram and TT videos also appear on a 
variety of meme pages while Twitter videos 
are less distributed 
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content from one 
video -> stitching 
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Russian Propaganda 
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Facilitators: Elena Pilipets & Marloes Geboers 
Tool-contributions by: Jason Chao & Stijn Peeters 
Designer: Emanuele Ghebaur 
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